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THE ROLE OF EVIDENCE IN THE IMPROVEMENT OF SCHOOL SYSTEMS 


Introduction 


Efficiency and equity have always been defining features of 
effective education policy in action. The global pandemic has 
further accentuated the importance of these themes. There is 
consensus among the global education community that the 
pre-Covid-19 learning crisis‘ and the extent of learning poverty? 
have deepened because of prolonged school closure. Prolonged 
lockdowns in many parts of the world have had a negative 
economic impact and funding constraints are placing strain on 
investment in education. The increased severity of the learning 
crisis coupled with new resourcing constraints make building 
back better an even greater challenge. The need for efficient and 
equitable spending which leads to good results is more important 
than ever. 


In economic terms, efficiency is concerned with ensuring the highest 
possible levels of output for a given level of input. With finite financial 
resources in the context of education policy, there is always a need 
for education ministers to be preoccupied with efficiency and 

value for money. When considering spending decisions and funded 


interventions, there should be a focus on steps needed to maximise the 


educational return on investment as measured by learning outcomes 
and other benefits to students and society. 


During the pandemic emergency, disadvantaged groups, including 
girls, have been hardest hit by remote learning shifts. Both 

the moral and the economic benefits of equity should not be 
underestimated. It is important to remember that using resources 
in an inequitable way is not efficient. Failing to equip disadvantaged 
students with foundational skills both damages their life chances 
and constrains economic development and societal wellbeing. It is 
efficient to be equitable. We know that investing in girls’ education, 
for example, can yield huge economic and social returns.3 The 
poor targeting of resources leads to wastage and the potential 
exacerbation of educational inequities. 


World Bank (2018) 

* World Bank (2019) 

3 Sperling, Winthrop & Kwauk (2015) 
4 Riggall et al (forthcoming) 


The linkage between equity and efficiency is often evident in 
approaches to funding and resource deployment or distribution. 
Itis not efficient to concentrate the most qualified and capable 
teachers in schools serving more advantaged communities. But if 
these higher-skilled teachers are deployed more evenly, they can 
be used not only to support more disadvantaged learners but also 
to support the professional development of other teachers, thus 
optimising their use. Smart targeting of resources to where they can 
do the greatest good is desirable, strategic and efficient. 


Efficient use of resources depends upon many factors. There are 
many forms of efficiency and inefficiency, but one key efficiency 
variable is the extent to which activities which require funding are 
designed and implemented in a way that is informed by relevant 
evidence. 


The application of insights about ‘what works, derived from robust 
research, combined with evidence about context and real-time 
system data have, when taken together, the potential to add 
substantial value to ‘building back better’ after Covid-19. Evidence 
is the fuel that drives smart, adaptive, impactful policy. At its best, 
the combination of knowing what the most effective investments 
are and having great system intelligence? that tells you how things 
are translating into action and change, will enable policy and 
decision-making to be strategic, open to fine tuning and deeply 
contextualised. 
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How should governments 
use evidence when making 
investment decisions’? 


Evidence-driven education reform is difficult. It can be hard to access relevant research and, 
compared to other sectors such as health, there is often a paucity of good quality research- 
based evidence that can be used to guide action. And when robust evidence has been generated 
in one country, there are often questions about how far the insights can be applied in the 
distinctive context of another country. Evidence comes in different forms. Senior officials need 
to know relevant research-based insights, but they also need easy access to other forms of 
evidence, including data about student outcomes disaggregated in terms of factors such as 
gender, geography, ethnicity, disability and family poverty levels. Policy should be responsive to 
‘user voice’ and the perceptions and concerns of students and parents constitute, therefore, an 
important form of evidence. 


There is an increasingly mature body of thinking about how we can encourage evidence uptake 
and knowledge mobilisation.! The demand for evidence (and the stimulation of demand through 
communications) are just as important as the supply of good quality evidence. 


Developing an evidence culture takes time. There is a need to identify concrete and manageable 
short-term priorities to improve the availability and use of evidence while setting ambitious 
long-term goals and plans in motion to build an evidence eco-system that will support long- 
term education improvement plans nationally and regionally. A systemic approach is needed 
that takes a broad view of evidence. Research that has been replicated in different contexts using 
experimental methods at scale constitutes the evidence ‘gold standard’ when considering the 
efficacy of different methods. However, there is also a place for rigorous qualitative research 
particularly when seeking to diagnose a problem. As the Building Evidence in Education group 
recently stated: 


‘When researchers want to know ‘what works’, 
quantitative methods are commonly selected 
instead of qualitative methods. However, without 
good qualitative data to contextualise these 
findings, ‘how or why things work’ can often 
remain obscured. ’? 


‘Hinton, Bronwin & Savage (2018) 
* DeJaeghere et al. (2020) 
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In addition to good quality quantitative and qualitative social science research, we should also 
encourage policymakers and practitioners to use other forms of non-academic evidence, 
including (for example) findings from school inspectors, trends apparent from national test 
results, and data from student voice surveys. There is a place for other forms of enquiry, such 

as rapid reviews of policy and practice based on the analysis of planning documentation.* Our 
own recent work’ suggests that to make the business of policymaking and implementation more 
evidence-informed, one needs a combination of evidence sources and wide view of what counts 
as evidence. 


We need to build ecosystems of evidence, recognising the importance of different forms of 
evidence. We also need to build the capacity to use evidence at every level of each education 
system so that different actors — headteachers, district officials and ministry HQ staff - are all 
Skilled at the analysis of evidence and take appropriate action based on the available evidence. 


We identify five key actions needed for an effective evidence-driven approach to education 
reform. 


FIGURE 1: Five components of an effective evidence-driven approach to education reform 


Ensure your data Use the available Align global insight Mediate Create collaborative spaces 
systems provide evidence about what about what works with evidence-based insights in which professionals at all 
disaggregated data and works and engage in the local contextual for policy professionals levels can engage with 
insight about student drive to build more of the evidence and need and frontline staff evidence and reflect on 
outcomes and resource __ right kind of evidence that practice — these can be 
distribution is relevant to you linked to specific 
improvement agendas — 
for example, girls’ 
education 


This view of the desirable components or actions is not a theoretical construct. All of these 
components already exist in different forms and different places. All are essential and they are all 
interrelated. 


In the following section, we exemplify the components with concrete case studies. 


3 For example, work commissioned and published by EdTech Hub since April 2020 has built knowledge of this kind. 
“ Gibbs, Thomas & Lucas (forthcoming) 
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Ensure your data systems 
provide disaggregated data and 
insight about student outcomes 
and resource distribution 


Almost every country has an Educational Management Information System (EMIS) and a linked 
national system for student assessment. GPE have rightly emphasised the relevance of the right 
approach to ‘big data’ in the management of investments in educational improvement. 


Data captured by EMIS provides policymakers with a form of evidence that should sit alongside 
research-based insights when spending decisions are made. Granular data about students and about 
resource distribution is a powerful tool. It can be used in many ways that are relevant to GPE: 


+ to target resources and professional development where need is great 

* to assist with accountability and improvement by identifying low-performing and high- 
performing schools and districts 

* to monitor the success of projects and interventions and support adaptive implementation so 
that methods can be modified in response to the analysis of real time data 

* to track the experience and outcomes of vulnerable groups of students, including girls. 


Evidence from data can be used in conjunction with evidence from research. If the data shows 
low enrolment rates for girls in a particular district, a rigorous qualitative social science enquiry 
can be used to reveal the reasons, according to out-of-school girls and their families, for non- 

attendance. 


Technology has an important role to play in data management and some systems are still overly 
reliant on data transfer through inefficient use of paper-based reporting. Some systems lack the 
ability to report at student level because there is no system of unique student identifiers within 
their EMIS.” As a result, it is much more difficult to track the possibly differential experience of 
girls and boys. 


‘Irving & Mitra (2019); GPE (2019a); GPE (2019b) 
* UNESCO-UIS (2020) 
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FIGURE 2: Countries using student-level data within EMIS? 


Sub-Saharan Africa Arab States Pacific South and West Asia East Asia 


22% > 63% : 67% ) 67% ) 


CASE STUDY: Building big data systems that help improve learning 
Sierra Leone's Education Data Hub 


5 Ibid. 


Since 2019, the government of Sierra Leone has been building an Education Data Hub, available online. This ‘flagship free quality 
education program relies on accurate data, state-of-the-art analytics and near real-time decision support tools: It combines and 
connects the Annual School Census and National Examination Results." It is an ambitious and radically new tool. Previously, there 
was no online sharing of official education data. It is an example of large-scale, national data systems that are being designed to 
support decision-making but also to provide accessible system intelligence for wider public use. Still in development, it provides 
an excellent learning opportunity too. The recent EdTech Hub-funded review of the Data Hub not only celebrates its purpose, 
ambition and development, but also identifies potential risks for others who are keen to embark on such a project. For example, 
technological considerations regarding access on different devices and the need for the data that sits within it to be kept up to 
date are two top requirements for the tool to continue to be useful and fulfil its purpose. The EdTech Hub report also highlights the 
requirement for data transparency and detail to enable trust and disaggregation.® 


4 Government of Sierra Leone (2017) 
5 EdTech Hub (2020) 
6 Freiermuth et al. (2020) 


CASE STUDY: Nepal equity index’ 


Nepal upgraded its EMIS to a web-based system in 2018, with unique school codes. Using this system, together with population 
census data, it was able to develop a composite educational equity index, based on indicators of access, retention and learning. 
The index covers equity dimensions of gender, location, caste and disability. The equity index is used, together with data on the 
distribution of education resources, to inform national and local level planning and budgeting in several ways: 


¢ To identify districts and municipalities in need of targeted support 

* To identify the drivers of inequity to inform policymaking (i.e. which groups of learners need additional support). 

¢ To identify municipalities that have poor outcomes and equity but higher-than-average resources and that therefore need more 
guidance and monitoring. 


7 Ministry of Education, Science and Technology in Nepal (2019) 
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Use the available evidence about 
what works and engage in the 
drive to build more of the right 
kind of evidence, relevant to you 


Although education is playing ‘catch up’ compared with health, there has been a great increase 
in the use of rigorous randomised control trials (RCT) studies in education in recent decades. 
One report identified over 1000 published RCT studies in education conducted 1980-2016.1 This 
expansion in quantitative research has, in turn, stimulated the development of meta-analyses 
that synthesise findings from studies considered to be methodologically sound. Research-based 
findings are becoming much more secure in two important areas: pedagogical techniques 
associated with good cognitive outcomes and aspects of neuroscience relevant to the work of 
schoolteachers. 


‘Connolly, Keenan & Urbanska (2018) 


CASE STUDY: Identifying ‘smart buys’ based on rigorous evidence 
review 


In October 2020, an international partnership known as the Global Education Evidence Advisory Panel (GEEAP) launched a paper 
examining cost-effective approaches relevant to the improvement of global learning and promoting the concept of ‘smart buys’: 
wise, evidence-based spending decisions.* This GEEAP report is based on the urgent need to know how to invest more efficiently 
to promote learning for all children. It is based on high-quality (where available) evidence that can comment on impact on 
learning and cost effectiveness. Findings are presented under four categories — great buys (cost-effective and supported by strong 
evidence base); good buys (good evidence that interventions are cost-effective); promising and low-evidence (small but rigorous 
studies that indicate cost-effectiveness, but overall evidence is limited); and bad buys (strong evidence that interventions have not 
worked and are not cost-effective). 


* Global Education Evidence Advisory Panel (2020) 
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The concept of ‘smart buys’ is clearly relevant to those responsible for education budgets. 
Policymakers have a duty to be guided when making spending decisions by relevant evidence 
and it is irresponsible to invest in ‘bad buys, even when there might be pressure from some 
interest groups to make ‘bad buys’ a spending priority. 


The work of GEEAP is an important step in the right direction, but there is much more to be done 
at global and national levels to strengthen the knowledge base. Too much of the global literature 
is derived from high-income country contexts. We need more rigorous research undertaken by 
Global South research organisations in low-income and middle-income countries. 


Itis also important to acknowledge that there are other types of reliable and relevant forms 

of evidence that can support spending decisions, even where the ‘best buys’ are yet to be 
identified. For example, in the field of girls’ education, we know that girl-focused approaches 
can be effective for the most marginalised and we know a great deal about the issues and 
challenges that may prevent access and completion of schooling. We know that the most 
marginalised girls are unlikely to be enrolled in school and will not be reached by interventions 
to improve school quality. Most out-of-school girls live in crisis-affected countries, and gender 
inequality in education tends to be greater in crisis-affected communities. Only 27% of refugee 
girls attend secondary school, compared to 36% of refugee boys; only 13% of girls living in the 
Lake Chad basin complete lower secondary school, compared to 20% of boys. Even in more 
stable contexts, groups of girls get left behind: girls with disabilities, young mothers, victims of 
child marriage, and girls from remote communities. 


Evidence suggests that governments should consider where girl-focused interventions are 
needed to ensure that all girls are able to participate in schooling and learn. Research we have 
conducted with FCDO, EdTech Hub and INEE suggests this might include: 


Clear communications to schools and communities to ensure that pregnant girls and 
young mothers are supported to continue learning (see for example, Sierra Leone and 
Kenya's response to ensuring that girls who have fallen pregnant during school closures are 
able to return to school) 

Safe spaces where girls can learn together. Girl-only spaces can support girls’ learning in 
emergencies.° Research in Kenya found that girls who attended community-based reading 
camps, or listened to radio lessons in groups during Covid-related school closure had higher 
learning levels than those that engaged in learning activities by themselves (using paper-based 
materials and/or radio and television lessons)® 

Coordination with other community-level services (for example, child protection and 
health) to follow up on cases of out of school girls at the household level. 


Our work acknowledges the importance of the role of evidence brokerage, school leaders and 
the teacher workforce, gender-sensitive pedagogy, community coalitions, the home and family 
and a broad systems approach to diagnosing challenges and designing solutions. 


3INEE (2021) 

* See Page & Naylor (2021) 
> INEE (2021) 

§ Amenya et al. (2021) 
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Align global insight about what 
works with local contextual 
evidence and need 


Simply taking insights from global research and attempting to apply this evidence is unlikely to 
work. Interventions require contextualisation and modification in the light of evidence at the 
level of the system. There is a need to design interventions that combine global insights with an 
understanding of the local situation, including information relating to potential context-specific 
barriers and risks to successful implementation. This fusion of global and local evidence was 
promoted by GEEAP. 


To be used effectively, therefore, this guidance should be combined with an assessment of 
context-specific needs and implementation constraints, including context-specific mechanisms, 
the quality of implementation, and political-economy constraints." 


Contextualisation goes beyond the design of interventions. If it is possible, interventions should 
be piloted so that modifications can be made based on evidence from the pilot. Similarly, 
implementation should be adaptive with changes made in response to issues that are identified 
during rollout. 


1 Global Education Evidence Advisory Panel (2020: 9) 


CASE STUDY: Applying and contextualising global insights in 
Rwanda 


To date, the Global Education Evidence Advisory Panel has identified only one ‘great buy’: the practice of giving information to 
parents about the benefits, costs and quality of education. An example of how this ‘great buy’ has been used in practice to get 
girls to return to school in Rwanda after Covid-19 comes from Education Development Trust's UK-Aid funded Building Learning 
Foundations programme.’ Rwanda was the first country to be approved for GPE funding. The Ministry's Covid-19 response plan 
included a focus on girls’ education and identified the specific challenge of drop-out caused by school closure. Initially, evidence 
was used to understand the barriers to returning to school after a crisis that required the closure of education establishments. 
Secondly, data was collected about which children were most vulnerable to drop-out and not returning to school, with the 
involvement of community committees. Then, building on the ‘great buy’ knowledge, information about the value of returning 
to school was shared through the school and community committees with families and parents. Finally, system data about 
enrolment was used to monitor the impact of the approach. The outcome identified through national data shows that between 
85% and 91% of girls in P1-5 returned when schools reopened. This compares to between 77% and 88% of boys. 


*Taken from the Education Development Trust's UKAid funded Building Learning Foundations Programme, Rwanda; full case study in Page & Naylor (2021) 
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CASE STUDY: Feedback from the frontline in Vietnam 


Young Lives data indicates that students in primary schools in Vietnam achieve good outcomes in foundational skills in 
comparison with other countries.> One study of the Vietnamese system suggests that policymakers in that country have 
successfully combined insights from global research with deep contextual knowledge. In recent years, reforms have been 
introduced based on global research relating to pedagogical practices, such as the use of diagnostic and formative assessment 
rather than numerical grades. District- and provincial-level officials see it as their role both to explain evidence-based reforms to 
schools but also to listen to concerns from school staff about the practicalities of implementation, so that the central ministry is 
well informed and can, when necessary, make modifications to the roll-out of the reforms. 


3 Boyden (2013) 
“ McAleavy, Ha and Fitzpatrick (2018) 
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Mediate evidence-based 
insights for policy professionals 
and frontline staff 


Policymakers and school staff are busy people who do not, for the most part, have the time to 
immerse themselves in the academic literature of educational research. Insights derived from 
research need to be summarised and communicated in a user-friendly way, with an emphasis 
on how findings can be translated into action. Education professionals may also need to be 
persuaded of the merits of new evidence-based methods if this requires them to abandon 
aspects of existing practice. Often reforms are quietly resisted by those who see no reason to 
change. Training, therefore, can play an important role in building both the skill and the will 
needed for the successful adoption of new methods. 


CASE STUDY: Helping professionals in Jordan and Lebanon to 
visualise evidence-based insights 


In both Jordan and Lebanon, Education Development Trust has been helping school improvement advisers to understand and 
apply insights from robust educational research. The advisers have access to a bank of ‘clinical practice cards’ which explain 
research findings in a highly visual and practical way, translating evidence into suggested practice. The cards cover findings from 
the academic literature relating to both teaching effectiveness and educational neuroscience. For example, the figures below 
show the cards explaining the principles of Assessment for Learning and how they can be applied in the classroom. 


Assessment for Learning and feedback xx i Examples 


Assessment for Learning (AfL) is a form of assessment that Primary/K — Grade 6 Example Secondary/Grade7 - 12 ample 
Puts students at the centre of the learning by using ‘formative During sing-a-long, the teacher monitors for The teacher explains to the whole class howto 
assessment’ (information about Knowledge and skills from ‘students who are not joining in or mouthing use conjunctions to join basic sentences together. 


assessment, alongside feedback on performance).' the words, Alter the singing the teacher divides After the teacher's explanation, the class workin 
the class into small groups and plays the music qroups and the teacher observes the students’ 
quietly. The teacher asks students to take turns performance, After group work, the teacher 
inpairs by singing one line from the song each —_reviews the work and then sets an informal quiz 
and moritors for which lines are challenging. The on conjunctions, The teacher realises from 
teacher identifies that the students only mouth these assessments that the students use ‘and’ 
the nouns and verbs and so gives feedback and and ‘but correctly however they can not use ‘or 
highlights articles and prepositions in future sing- appropriately. The teacher reflects on how ‘or’ 
along activities, ‘was taught, includes a feedback session on ‘or’ 
and adapts or adjusts the lesson and their lesson 
plan accordingly. 


# Effective AfL is integrated into teaching and learning so 
students understand “what they are leaming and why’ {leaming 
goals}, how learning is assessed and the criteria used? 

¢ Formative assessment helps students to leam more *4 

of Feedback is more beneficial when given as soon as possible 
with the focus on the task and what students should do next to 
improve? 

J AfLis a personalised approach to assessment, as current 
pertormance is compared to previous levels, so learning gaps 
are identified and new learning goals set® 

# Ensure students know and understand the details of their 
performance and learning goals, so they are involved and can 
self monitor pragress 

J Inthe classroom, AfL is supported by effective questioning, 
regular teedback on assessed work, co-constructed leaming 
goals, peer and self assessment and informed planning for 
learning? 


References 

1 Back P-andWitam D (1938) Insite 

2 SwatfieldS (2019 Gettingto the tear pes, Policyand Practice 144), 435-449 
Rounlede. 


H (2007 
6 No.JandSammone P 2019 Ertectwe teaching:a fevew of research and eucence, Reading: CBT Education Trust 
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Create collaborative spaces in 
which professionals at all levels 
can engage with evidence and 
reflect on practice 


Introducing evidence-based reforms to improve educational outcomes is an exercise in change 
management. Adoption and effective implementation of proposed changes are unlikely to 
happen simply because the changes have been formally agreed or mandated. Professionals 
at all levels of the education system benefit from opportunities to discuss the challenges of 
change with peers in a supportive, collaborative setting. In the country examples given above, 
professionals were often given a forum for professional dialogue and problem-solving. In 
Rwanda, communities of practice existed at school level and headteachers were part of a 
community of practice at community level. These networks were used as a place in which the 
plans for ensuring that girls returned to school post-pandemic were discussed. Similarly, in 
Vietnam, reforms to the assessment system based on international research were discussed 
within the collaborative ‘subject groups’ of teachers that exist in all Vietnamese schools. 


The following case study shows how the same idea of how collaborative problem-solving 
engagement, prompted by evidence and a formal requirement to act, can lead to real change. 


CASE STUDY: Helping ‘out-of-school’ students in South East Asia 


Several research studies, especially studies conducted by UNICEF, convinced policymakers in several South East Asian countries 
that there was an urgent need to tackle the problem of Out-of-School Children and Youth (OOSCY). They decided to address this 
collectively, under the aegis of ASEAN, and signed a joint declaration in 2016. Further evidence-based insights were provided 
by UNESCO Bangkok, based on research into effective policies and programmes and an analysis of the economic implications 
of OOSCY. This evidence was sought to support decision-making and design change programmes that had at their heart a 
collaborative problem-solving partnership with members from different ASEAN countries which were committed to rapid 
progress in ensuring school enrolment. The Asia Regional Network on Out of School Children was established as a platform 

to disseminate knowledge and deliver capacity-building activities for country governments and NGOs, including networking 
events, workshops and one regional summit to showcase innovative programmes on OOSCY. Their report, ‘Leveraging regional 
collaboration to enhance impact’? documents an impressive story of evidence prompting commitment to act and leading to 
collaborative problem-solving and change. 


1 Education Above All, Education a Child & Results for Development (2020) 
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Conclusion 


In this paper, we have argued for the necessity of evidence-informed educational leadership. 
Educational decisions determine life chances and are too important to be left to chance. The 
magnitude of today’s learning crisis is great and requires a highly professional response. We see 
evidence as the ‘fuel’ that can power effective action. Evidence comes in many different forms 
and mature education systems recognise the value of a range of evidence types, including 
academic research, disaggregated data from the frontline and student voice. Worldwide, the 
students who are least likely to go to school and unlikely to excel at school are those who have 
disadvantaged backgrounds. Access to the right types of evidence is necessary but not sufficient 
as a means of tackling the equity challenge. Education systems will continue to fail vulnerable 
students unless leaders at all levels have the skill and the will to make good use of evidence. 
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